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Abstract 

A medical school conducted a summer pre-matriculation program. The program provided basic sciences content 
comparable to first year medical student instruction along with clinical and other learning experiences. The study 
purpose was to examine self-confidence levels and reasoning skills of a single cohort of students. We examined the 
association between students’ confidence, reasoning skills and performance measures. We also examined performance 
differences based on student gender, ethnicity and socioeconomic backgrounds. 32 students were given two confidential 
surveys in a pre/post-test format. Student confidence was measured using a modified version of the Motivated 
Strategies for Learning Questionnaire (MSLQ). Student reasoning skills were measured using the Health Sciences 
Reasoning Test (HSRT). Student performance was measured via written exams. Survey item mean differences and 
inter-item correlations were examined using the T-test and Spearman procedures. Data were analyzed using SPSS 
software, with significance levels for all tests set at p=.05. Completed pre- and post-test surveys were returned from 29 
students (91%) for the MSLQ and 18 students (56%) for the HSRT. A single item on the MMSLQ was significantly 
different on the post-test. Student race, gender and socioeconomic background were poor predictors of overall 
performance but were associated with selected measures of student confidence and reasoning skills. Students enrolled in 
medical school preparatory classes may be overly confident prior to experiencing medical school. Demographic 
variables deserve attention by medical school staff, particularly when the school’s mission focuses on successful 
performance by students from under-represented groups. 
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1. Introduction 

The Summer Program for Future Doctors (SPFD) at the Brody School of Medicine, East Carolina University was 
established in 1978 with the purpose of preparing students to attend medical school, in support of the school’s mission 
and the overall healthcare workforce needs of the state of North Carolina (Hardy, 1999). SPFD is a nine-week program 
that enrolls 20-30 students per year. Most participants are juniors or seniors in college who are interested in applying to 
medical school or post-baccalaureate students who may have been unsuccessful in an attempt to gain admission to 
medical school (pre-matriculating students). A small number of students are admitted to the program after acceptance to 
medical school but prior to actual enrollment (matriculating students). The course of study allows students to strengthen 
their knowledge base in the sciences, improve their overall critical thinking skills, and demonstrate their aptitude for the 
rigorous demands of formal training in medicine. The curriculum features approximately 220 hours of instruction. Basic 
science courses include biochemistry, anatomy, physiology, and neuroscience. The curriculum also provides instruction 
in learning strategies, communication skills and behavioral sciences; features shadowing experiences with physicians 
and exposure to clinical skills in both simulated and “real time” environments; and offers formal preparation for the 
medical school admissions process including mock student interviews and Medical College Admissions Test (MCAT) 
preparation skills. All applicants to the program must be legal residents of North Carolina. The SPFD program is 
popular statewide, usually attracting between 150-200 applicants each year. The application process focuses on 
attracting students from under-represented minorities (URM) and non-traditional groups, as defined using both ethnic 
and socioeconomic criteria. Medical school faculty members teach all basic science courses without compensation, 
which represents a sizeable contribution of time during the summer. Rising second- year medical students serve as 
Teaching Assistants (TA) and receive a small stipend for their role in teaching and mentoring SPFD students. 
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It was previously reported by Hardy (1999) that students who have demonstrated strong performances and consistent 
improvement in the summer program are likely to gain admission to medical school and perform satisfactorily. Within 
medical education, the emphasis on recruiting students from a variety of under-represented groups and offering them 
academic preparation for medical school has evolved over time and now appears to be firmly established as a viable 
strategy to help increase diversity within health professions training programs. The Association of American Medical 
Colleges has been a national leader is establishing programs to increase the number of under-represented minority 
students who attend medical school (AAMC 2014). According to Pechura (2001) funding for many programs of this 
type was furnished beginning in the 1970s by the Robert Wood Johnson Foundation. By 1991 nearly half of all medical 
schools offered some type of pre-professional training programs designed to increase the representation of URM 
students (Cantor, Bergeisen and Baker, 1998). Although limited evidence exists concerning the effectiveness of these 
programs, there is some indication that they have proven successful in increasing minority student participation in 
medical school (Pechura, 2001; Cantor et al, 1998; Hesser, Cregler and Lewis, 1998; Andrioloe and Jeffe, 2011; Deas, 
et al., 2012) and achieving diversity-related educational outcomes (Saha, Guiton, Wimmers and Wilkerson, 2008). 

Once students from under-represented backgrounds are admitted to medical school, what are the factors that contribute 
to or hinder their success? Again, there is not a plethora of evidence in regard to this question. Two early studies by 
Strayhorn (1999 and 2000) demonstrated that ranked performance data from participation in structured summer 
premedical programs were useful in predicting subsequent performance at a single medical school. Subsequent studies 
have shed additional light on whether skills acquired during these types of programs have resulted in higher rates of 
achievement by participants (Tekian and Hruska, 2004; Odom, Roberts, Johnson and Cooper, 2007; Orom, Semalulu 
and Underwood, 2013). Some studies include data from students in under-represented groups wherein they discuss the 
positive as well as negative aspects of entering a formal course of study in medicine or other health professions, and 
many students have shared in a powerful manner the obstacles encountered along the way (Odom et al., 2007). More 
recently, accrediting bodies have placed more emphasis on the learning environment of medical schools and health 
systems providing physician training, resulting in a more direct connection between how learning happens in medical 
school and the successful achievement of educational outcomes (Liaison Committee on Medical Education, 2015; 
Accreditation Council for Graduate Medical Education, 2015). One study showed that selected changes in the medical 
school learning environment leads to improved academic outcomes (Lieberman, Frye, Thomas, Rabek and Anderson, 
2008). 

Although the SPFD has had a history of success, we wanted to delve further into the issue of what the student 
experience was like within our program. Specifically, we sought to determine whether student performance during the 
SPFD was impacted by demographic characteristics as well as by self-reported student confidence and measurable 
cognitive reasoning skills. Demographic characteristics (e.g., ethnicity, gender, socioeconomic background) are widely 
thought to be related to admission to medical school (Benbassat and Baumal, 2007) and subsequent performance (Woolf, 
Potts and McManus, 2011). Confidence is also thought to impact physician performance, with particular reference to 
one’s ability to manage uncertainty and ambiguity and willingness to engage in specific learning activities (Blanch, Hall, 
Roter and Frankel, 2008; Haimowitz, 2013; Clanton, Gardner, Cheung, Mellert, Evancho-Chapman and George, 2014). 
And, the measurement of reasoning skills is generally viewed as important to physician training, particularly regarding 
the specific abilities of physicians to use clinical reasoning and critical thinking in the performance of their patient care 
duties. In conceptualizing this study, we felt that an increased understanding of student experience in the SPFD (as 
assessed using instruments designed to measure student confidence and student reasoning skills) would not only result 
in programmatic improvements but could also potentially shed light on how and why programs that seek to prepare 
students for medical school are successful. 

2. Methods 

This study was approved by our institutional review board. A total of 32 students admitted to the 2013 iteration of the 
SPFD were given two confidential surveys in a pre/post-test format. Each student was assigned a confidential 
identifying number for the purpose of matching data from the pre- and post-test surveys. Paper and pencil surveys were 
administered on day two of the program and on the last day of activities; online surveys were administered during 
designated periods during the first and last week of the program. Of the 32 students admitted to the SPFD, 14 students 
(44%) were from URM groups. A total of 14 students (44%) were from North Carolina designated Tier 1 and Tier 2 
counties identified as “socioeconomically distressed” and 18 students (56%) were from North Carolina designated Tier 
3 counties that were not identified as socioeconomically distressed. 

Student confidence was measured using a paper and pencil format modified version of the Motivated Strategies for 
Learning Questionnaire (MSLQ), with the creation of this instrument described by Duncan and McKeachie (2005). The 
original version of the Motivated Strategies for Learning Questionnaire consists of 81 items organized in two sections 
which focus on student motivation and student learning strategies. This instrument has been previously modified and 
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tested with medical students and resident physicians to investigate its psychometric properties and to explore potential 
associations between student motivation, learning strategies and performance (Stegers-Jager, Cohen-Schotanus and 
Themmen, 2012; Turan and Konan, 2012; Cook, Thompson and Thomas, 2011). The rating scale is a 7-point Likert 
type ordinal scale. The instrument is best used within the context of a specific course of instruction (Cook et ah, 2011). 
We modified the questionnaire to reflect what, in our judgment, were the unique characteristics of the SPFD. 
Specifically, we selected a total of 16 items from the original MSLQ and modified them to fit program evaluation needs 
for the SPFD as shown in Table 1. 

Table 1. Modified motivated strategies for learning questionnaire (MSLQ)* 

Directions: rate each of the following items based on your learning/study behaviors. Your rating should be on a 7-point 
scale where 1 = not at all true of me to 7 = very true of me 

Item 1 I prefer class work that is challenging so I can learn new things 
Item 2 I expect to do well overall in the SPFD 

Item 3 I am so nervous during an exam that I cannot remember facts I have learned 

Item 4 I am sure that I can understand the ideas taught in the SPFD 

Item 5 Compared with others doing the SPFD, I think I am a good student 

Item 6 I often choose to write papers on topics I will learn something from, even if they require more work 

Item 7 I am sure that I can do an excellent job on the problems and tasks assigned during the SPFD 

Item 8 Even when I do poorly on an exam, I try to learn from my mistakes 

Item 9 My study skills are excellent 

Item 10 I feel that I already know a great deal about the subject matter covered during the SPFD 

Item 11 I worry a great deal about exams 

Item 12 It is hard for me to decide what the main ideas are in what I read 

Item 13 I always try to understand what the teacher is saying, even if it doesn’t make sense 

Item 14 Even when study materials are dull and un-interesting, I keep working until I finish 
Item 15 Before I begin studying I think about the things I will need to do in order to learn 

Item 16 I often find that I have been reading for class but don’t know what the materials are all about _ 

(*modified from the original version for program evaluation purposes.) 

The process of selecting these 16 items was somewhat arbitrary, but was done by the co-authors collaboratively. We 
acknowledge that the selection of these particular items may represent our own research intuitions, rather than a 
description of a meaningful sub-set of the overall instrument item set; and that our chosen conceptual framework in 
selecting these 16 items may unavoidably influence data interpretation. We made no attempt in this study to validate our 
selected items as a separate sub-scale of the 81-item MSLQ; rather, our focus was related to our interest in 
understanding how SPFD students experience the program itself. 

Student reasoning skills were measured using the Health Sciences Reasoning Test (HSRT), a commercially available 
instrument which can only be administered online and measures reasoning and decision-making processes (Health 
Sciences Reasoning Test, 2015). The HSRT provides both an overall performance score and detailed analyses scores 
based on five domains (induction, deduction, analysis, inference and evaluation sub-scores). According to its developers, 
the test is specifically calibrated for trainees in health sciences educational programs (undergraduate and graduate) and 
for professional health science practitioners. Scores on this instrument have been found to predict successful 
professional licensure and high clinical performance ratings (HSRT, 2015). Again, our primary interest in using the 
HSRT was not connected to instrument reliability or predictive validity but rather to gain insight into how the SPFD 
program may have impacted students’ reasoning skills. In other words, we envisioned using data from both the modified 
MSLQ and the HSRT primarily as a function of program evaluation (as opposed to diagnostic application of results to 
individual SPFD students). 

Student performance in the basic science courses taught during the SPFD was measured via a total of twelve written 
exams. Two of the courses also had weekly quizzes that were averaged to provide the equivalent of an additional exam 
in each course. A final overall SPFD mean score was computed for each student, based on the average of each student’s 
final average in each course. Survey item mean differences and inter-item correlations pertaining to student confidence 
were examined using the T-test and Pearson procedures respectively. For item mean comparisons, we compared items 
within each group for both pre- and post-test surveys, and also within groups across the pre- and post-test surveys. For 
example, for analyses based on student gender we compared how female students rated items on the pre-test and 
post-test surveys and also compared how female and male students rated items on both pre- and post-test surveys. We 
also performed correlational analyses using data from the final overall SPFD average exam score and student 
demographic data, looking for potential associations between these items. Unfortunately, correlation between overall 
SPFD exam score and student reasoning skills could not be computed, as the commercial software company did not 
offer this type of analysis. Data were analyzed using SPSS software version 22, with significance levels for all tests set 
at p=.05. 
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3. Results 

Of the total number of students enrolled in the SPFD (N=32), completed pre- and post-test surveys were returned from 
29 (91%) for the modified MSLQ, but from only 18 (56%) for the HSRT. While paper and pencil formatted surveys 
were distributed and collected in person for the modified MSLQ, students had to complete the HSRT instrument via 
online platform within a certain window of time at the beginning and end of the SPFD experience. We speculate that the 
online formatting of the HSRT may have impacted the completion rate in a negative manner. The resulting sample sizes, 
while small, were considered to be useful for program evaluation purposes. 

3.1 Student Confidence (Modified MSLQ) 

The MSLQ was used to measure student confidence by comparing pre- and post-test surveys administered as previously 
described. Across all 29 students who completed both pre- and post-test surveys, a single item (“I expect to do well 
overall in the SPFD”) was significantly different when comparing pre-test (mean=6.21) and post-test results 
(mean=4.93, p=0.001), with students expressing less overall confidence at the end of the SPFD program. 

Regarding student gender, a single confidence item (“I am sure that 1 can understand the ideas taught in SPFD”) was 
significantly different only on the post-test with 14 female students reporting lower confidence (mean=5.29) compared 
to 15 male students (mean=6.27, p=0.007); there was no significant difference on the pre-test for this item. No other 
items were significantly different based on student gender. 

Regarding student socioeconomic background, 12 students from more economically distressed counties scored 
significantly higher on a single confidence item (“I expect to do well overall in the SPFD”) when comparing the pre-test 
(5.92) and post-test (4.58, p=0.04) means. And, 17 students from less economically distressed counties had a similar 
result on this same item when comparing the pre-test (6.38) and post-test (5.25, p=0.02) means. No other items were 
statistically significantly different based on students’ socioeconomic background. 

Regarding student ethnicity, a single confidence item (“I often choose to write papers on topics I will learn something 
from, even if they require more work”) showed a significant difference on the pre-test survey when comparing 17 
non-minority students (mean=5.35) and 12 minority students (mean=3.82, p=0.007). On the post-test survey, there were 
three confidence items which showed significant differences. On two of these items, the non-minority students rated 
items higher on the post-test: item four, “I am sure that 1 can understand the ideas taught in the SPFD” (mean=6.23 
versus 5.27, p=0.02); and item 6, “I often choose to write papers on topics 1 will learn something from, even if they 
require more work” (mean=5.88 versus 4.18, p=0.007). On the third item (“I often find that I have been reading for 
class but don’t know what the materials are all about”), minority students rated this item higher (mean=4.00 versus 2.77, 
p=0.05). No other items were significantly different based on student ethnicity. 

3.2 Student Reasoning (HSRT) 

The HSRT was used to assess student reasoning skills based on five domains (induction, deduction, analysis, inference 
and evaluation sub-scores). We compared pre- and post-test survey results in a similar manner as previously described; 
however, our sample size was considerably smaller. Due to the smallness of our sample size, we report only results for 
the overall HSRT reasoning score (and not for the five sub-domain scores). Across all 18 students who completed both 
pre- and post-test surveys, there were no items that were significantly different when comparing pre-test reasoning 
scores (mean overall score=22.1, 63 rd percentile, standard deviation=29.7) versus post-test reasoning scores (mean 
overall score=22.6, 67 th percentile, standard deviation=30.1, p=0.075). Scores on the post-tests represented a slight 
upward trend on overall clinical reasoning and critical thinking skills by SPFD students as measured by this instrument. 

Regarding student gender, 8 male students and 10 female students completed the pre- and post-test of reasoning skills. 
On the pre-test, male students scored significantly higher (mean overall score=25.1, 83 rd percentile, standard 
deviation=2.6) than female students (mean overall score=19.7, 48 th percentile, standard deviation=4.4, p=0.007). 

Regarding student socioeconomic background, 3 students from more economically distressed counties and 15 students 
from less economically distressed counties completed both pre- and post-tests of reasoning skills. There were no 
significantly different means found on any of the items when comparing these two groups. Performance by students 
from more economically distressed counties was slightly lower on the post-test (mean overall score=22.7, 65 th 
percentile, standard deviation=7.6) than on the pre-test (mean overall score=23.3, 66 th percentile, standard 
deviation=6.1). However, performance by students from less economically distressed counties was slightly higher on 
the post-test (mean overall score=22.6, 68 th percentile, standard deviation=4.9) than on the pre-test (mean overall 
score=21.9, 63 ld percentile, standard deviation=4.1). Scores on the post-test represented a slight upward trend on overall 
clinical reasoning and critical thinking skills by students from less economically distressed counties, when compared 
with students from more economically distressed counties where a slightly lower trend was observed. 

Regarding student ethnicity, 8 non-minority students scored significantly higher on the reasoning pre-test (mean overall 
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score=25.0, 83 ld percentile, standard deviation=3.4) than 10 minority students (mean overall score=19.8, 49 th percentile, 
standard deviation=4.1, p=0.01). When comparing these two groups performance on the post-test, however, the 
difference was not statistically significant (p=0.06) but the same trend was present: non-minority students scored higher 
(mean overall score=25.0, 83 rd percentile, standard deviation=2.4) than minority students (mean overall score=20.7, 55 th 
percentile, standard deviation=5.6). When comparing the performance of minority students on the pre-test (mean overall 
score=19.8, 49 th percentile, standard deviation=4.1) versus post-test (mean overall score=20.7, 55 th percentile, standard 
deviation=5.6) there was an upward trend observed. When comparing the performance of non-minority students on the 
pre-test versus post-test, the results were virtually identical (mean overall scores of 25.0 on both, percentiles 81.2 versus 
83.2, standard deviation 3.4 versus 2.4). 

3.3 Overall Performance on SPFD Exams 

Final average basic science exam scores (i.e., average score across all courses) ranged from 95.5 to 42.9, with a median 
score of 80.4. Female students (80.9, SD=11.6) performed better than male students (75.7, SD=13.6); non-minority 
students (84.5, SD=7.9) performed significantly better than minority students (74.0, SD=13.8, p=0.03); and students 
from more economically distressed counties (79.9, SD=12.1) performed better than students from less economically 
distressed counties (76.9, SD=14.0). 

We also examined the association between demographic variables and overall exam performance by performing 
correlational analyses. Regarding student gender, there was no significant correlation with final grade (R=0.16, p=0.42), 
indicating that there was no association between student gender and overall exam performance. Regarding student 
socio-economic background, the association between student socio-economic background and exam performance was 
negative and not significant (R=-0.07, p=0.70), indicating that there was no association between a given student’s 
overall exam performance based on the economic conditions of his/her home county. Regarding student ethnicity, the 
association between ethnicity and exam performance was negative and not significant (R=-0.30, p=0.13), indicating that 
there was no association between a given student’s ethnicity and overall exam performance. 

4. Discussion 

In this pilot study of an intense summer program designed to prepare students for admission to medical school, we 
sought to further our understanding of the impact of selected student characteristics on student performance, as an 
additional method of program evaluation. By doing so, we hoped to gain further insight into how we can support SPFD 
students in terms of their academic success and further enhance their opportunities to be accepted into the medical 
school of their choice. 

Our results indicate that student race, gender and socioeconomic background were not associated with overall 
performance during the SPFD based on analyses of a single cohort of students. This is reassuring in the sense that one 
would expect senior level college students who have been on the pre-medical school academic pathway to be relatively 
high functioning individuals regardless of background. Interestingly, however, those same demographic factors were 
significantly associated with selected measures of student performance, confidence and reasoning skills. 

In regard to student self-reported confidence, we had anticipated that experiencing a program designed to be very 
similar to the first year of medical school would provide a boost in confidence for the majority of students. Therefore, it 
was somewhat surprising to us that, as measured on survey item number 1 (“I expect to do well overall in the SFPD”), 
we found that SPFD students as a group rated their own confidence lower at the end of the SPFD than at the beginning. 
Based on both this result and our everyday experience with SPFD students over a number of years, we speculate that 
some students who enroll in a medical school preparatory experience may be somewhat over-confident prior to 
experiencing a realistic medical school academic environment. After all, many such students have excelled in the 
classroom previously, and thus may not fully appreciate the extent to which the academic rigors of medical school are 
different than they expected. 

In regard to student reasoning skills, the finding that the reported scores on an objective external measure increased 
slightly but not significantly was also somewhat surprising. It may well be that students at this stage of their educational 
journey have already maximized the development of their general reasoning and critical thinking skills, at least as 
measured by this particular instrument. Alternatively, it may be that the use of the chosen external instrument to 
measure these skills in this particular type of student population is not optimal. This is an issue that deserves further 
study. 

Our results also seem to indicate that demographic variables deserve attention by medical educators particularly when 
the school’s mission focuses on successful performance by students from under-represented groups. Regarding student 
gender, our results indicated that male students were more self-confident at the time of post-test survey and scored 
significantly higher in reasoning skills at time of pre-test. Flowever, male students performed lower (but not 
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significantly so) on knowledge-based exams. Measures of association revealed no significant overall association 
between student gender and final exam grade average across the entire student group. These findings are consistent with 
a variety of studies from the medical education literature indicating that gender-based differences may exist in medical 
student performance and/or attitudes (Blanch et al., 2008; Ramsbottom-Lucier, Johnson and Elam, 1995; Haist, Wilson, 
Elam, Blue and Fosson, 2000; Osman, Schonhardt-Bailey Walling, Katz and Alexander, 2014). In particular, we find it 
interesting that male students were more confident, but female students performed at higher levels on knowledge-based 
exams. 

Similarly, we also found that student ethnicity is an important variable to consider in terms of potential impact on 
educational processes and/or achievement. Measures of association revealed no significant overall association between 
student ethnicity and final exam grade average across the entire student group. However, our results indicated that 
students from minority groups were less confident overall and achieved lower scores on reasoning skills than did 
students from non-minority groups. Students from minority groups also scored significantly lower on knowledge-based 
exams than did students from non-minority groups. These findings are consistent with other studies wherein the 
performance of students from minority groups during medical school is found to be lower, thereby indicating a need for 
strong academic support services for minority medical students (Deas et al., 2012; Odom et al., 2007). These findings 
were not surprising to us, given the long history of our school’s involvement in offering medical education to students 
from under-represented groups and our excellent level of academic support services consistent with that mission. What 
seems clear is that students from under-represented groups may need support earlier in the educational process of 
gaining entrance to medical school. Enhancing pre-admission opportunities for such students, and early identification of 
potential academic difficulties, seem indicated if our overall policy goal in medical education is to expand the number 
of physicians from under-represented groups. 

In regard to student socio-economic background, we found that students from economic areas categorized as more 
distressed did not reveal any significant differences in either confidence levels or reasoning skills at the beginning or 
end of the summer program. In fact, students from more economically distressed counties performed slightly higher 
than students from less distressed counties on knowledge-based exams; and measures of association revealed no 
significant overall association between student socio-economic background and final exam grade average across the 
entire student group. There are indications in the medical education literature that students from more rural, less 
populated areas are more likely to eventually practice as physicians in those same areas (Owen, Conaway, Bailey and 
Hayden, 2007; Gill, McLeod, Duerksen and Szafran, 2012). We found these results reassuring for our efforts to recruit 
medical students from portions of our state where, at first glance, one might be tempted to believe that students would 
not perform as well as their counterparts from larger, more prosperous areas. Again, if it is a policy goal to enhance 
opportunities for such students to pursue a medical education in order to address physician shortages in rural areas 
suffering from economic distress, our study would seem to lend at least limited support to this goal. 

5. Conclusion 

The importance of recruiting students from underrepresented minority groups and economically distressed geographic 
areas to enter the field of medicine has been discussed in the literature as a viable strategy to help address general 
physician workforce issues, with specific reference to increasing the supply of physicians in rural and other areas where 
there appear to be severe shortages of health care providers (Figueroa, 2014). The Summer Program for Future Doctors 
(SPFD) at Brody School of Medicine, East Carolina University was established over thirty years ago with these 
purposes in mind and in support of the overall primary care mission of the school. A more in-depth analysis of the 
SPFD from a program evaluation perspective provided us with general support for our efforts while simultaneously 
revealing factors (student demographics in particular) that may impact student success. These insights lend general 
support to pre-matriculation preparation programs for future medical students while also illuminating potential issues 
that such programs should consider in order to maximize their effectiveness. 
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